Electronic apparatus and method for providing manual thereof

ABSTRACT

A method for providing a manual of an electronic apparatus is provided. The method includes providing a display device with one of a plurality of manuals for an external device, obtaining a user response for the provided manual, determining an understanding level for the manual based on the obtained user response, and substituting the provided manual with another manual among the plurality of manuals according to the understanding level. The plurality of manuals may include a general manual, a detailed manual, and a simple manual.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is based on and claims priority under 35 U.S.C. §119(a) of a Korean patent application number 10-2019-0119099, filed onSep. 26, 2019, and a Korean patent application number 10-2019-0113790,field on Sep. 16, 2019, in the Korean Intellectual Property Office, thedisclosure of each of which is incorporated by reference herein in itsentirety.

BACKGROUND 1. Field

The disclosure relates to an electronic apparatus and a method forproviding a manual. More particularly, the disclosure relates to anelectronic apparatus for providing a manual according to a user responseand a method for providing a manual thereof.

2. Description of Related Art

The development of electronic technology has enabled a user to beprovided with various content at any time through a user terminaldevice. The user terminal device may receive or store various types ofcontent, such as images, voice, and video, as well as a text, from anexternal device, and may provide the same to a user.

In particular, recently, content such as instructions or guides forproviding a user with information are also provided as content such asan image, voice, a video, or the like, in order to facilitateunderstanding of the user, and the user can more intuitively and easilyobtain information by using content such as an image, voice, a video orthe like.

However, in that the content is delivered unilaterally to the user, thatis, the content is provided to the user without consideration of theuser response or user's understanding level of the content, the user mayfeel inconvenient to navigate (e.g., forward, rewind, stop, etc.) theprovided content or search other content if the provided content is notsuitable for the user.

The above information is presented as background information only toassist with an understanding of the disclosure. No determination hasbeen made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the disclosure.

SUMMARY

Aspects of the disclosure are to address at lease the above-mentionedproblems and/or advantages and to provide at least the advantagesdescribed below. Accordingly, an aspect of the disclosure is to providean electronic apparatus for determining a user's understanding level fora manual provided to a user and providing the user with a suitablemanual based on the determined understanding level, and a method forproviding a manual thereof.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of the presented embodiments.

In accordance with an aspect of the disclosure, a manual providingmethod of an electronic apparatus is provided. The manual providingmethod includes providing a display device with one of a plurality ofmanuals for an external device, obtaining a user response for theprovided manual, determining an understanding level for the manual basedon the obtained user response, and substituting the provided manual withanother manual among the plurality of manuals according to theunderstanding level, wherein the plurality of manuals include a generalmanual, a detailed manual, and a simple manual.

In accordance with another aspect of the disclosure, an electronicapparatus is provided. The electronic apparatus includes a communicationinterface, a memory, and a processor configured to control thecommunication interface to provide a display device with one of aplurality of manuals for an external device, obtain a user response forthe provided manual, determine an understanding level for the manualbased on the user response, and substitute the manual with anothermanual among the plurality of manuals according to the understandinglevel, and the plurality of manuals include a general manual, a detailedmanual, and a simple manual.

Other aspects, advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses various embodiments of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the disclosure will be more apparent from the followingdescription taken in conjunction with the accompanying drawings, inwhich:

FIGS. 1, 2, 3, 4, 5, and 6 are flow charts to describe a method forproviding a manual of an electronic apparatus according to variousembodiments of the disclosure;

FIG. 7 is a view to describe a content providing system for providing acontent according to a user's understanding level according to anembodiment of the disclosure;

FIG. 8 is a block diagram illustrating a configuration of an electronicapparatus according to an embodiment of the disclosure;

FIG. 9 is a block diagram illustrating a dialogue system of anelectronic apparatus according to an embodiment of the disclosure;

FIG. 10 is a view to describe a plurality of manuals according to anembodiment of the disclosure;

FIG. 11 is a view to describe a display device for selecting asub-content to be reproduced among a plurality of sub-contents relatedto an external device according to an embodiment of the disclosure;

FIG. 12 is a view to describe an electronic apparatus for determining anunderstanding level from user voice information according to anembodiment of the disclosure;

FIGS. 13 and 14 are views to describe an electronic apparatus fordetermining a reproduction starting point of a substitute manualaccording to various embodiments of the disclosure;

FIGS. 15, 16, and 17 are views to describe an electronic apparatus forreceiving a manual transmission command from the display deviceaccording to various embodiments of the disclosure;

FIG. 18 is a sequence view including an electronic apparatus accordingto an embodiment of the disclosure;

FIG. 19 is a view to describe an electronic apparatus according to anembodiment of the disclosure; and

FIG. 20 is a block diagram illustrating an electronic apparatusaccording to an embodiment of the disclosure.

Throughout the drawings, it should be noted that like reference numbersare used to depict the same or similar elements, features, andstructures.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of variousembodiments of the disclosure as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding, but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the various embodiments describedherein can be made without departing from the scope and spirit of thedisclosure. In addition, descriptions of well-known functions andconstructions may be omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but are merely used by theinventor to enable a clear and consistent understanding of thedisclosure. Accordingly, it should be apparent to those skilled in theart that the following description of various embodiments of thedisclosure is provided for illustration purposes only and not for thepurpose of limiting the disclosure as defined by the appended claims andtheir equivalents.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

In this document, the expressions “have,” “may have,” “including,” or“may include” may be used to denote the presence of a feature (e.g., anumerical value, a function, an operation), and does not exclude thepresence of additional features.

In this document, the expressions “A or B,” “at least one of A and/orB,” or “one or more of A and/or B,” and the like include all possiblecombinations of the listed items. For example, “A or B,” “at least oneof A and B,” or “at least one of A or B” includes (1) at least one A,(2) at least one B, (3) at least one A and at least one B all together.

The terms such as “first,” “second,” and so on may be used to describe avariety of elements, but the elements may not be limited by these terms.The terms are labels used only for the purpose of distinguishing oneelement from another.

It is to be understood that an element (e.g., a first element) is“operatively or communicatively coupled with/to” another element (e.g.,a second element) is that any such element may be directly connected tothe other element or may be connected via another element (e.g., a thirdelement). On the other hand, when an element (e.g., a first element) is“directly connected” or “directly accessed” to another element (e.g., asecond element), it can be understood that there is no other element(e.g., a third element) between the other elements.

The expression “configured to” can be used interchangeably with, forexample, “suitable for,” “having the capacity to,” “designed to,”“adapted to,” “made to,” or “capable of” The expression “configured to”does not necessarily mean “specifically designed to” in a hardwaresense. Instead, under some circumstances, “a device configured to” mayindicate that such a device can perform an action along with anotherdevice or part. For example, the expression “a processor configured toperform A, B, and C” may indicate an exclusive processor (e.g., anembedded processor) to perform the corresponding action, or ageneric-purpose processor (e.g., a central processing unit (CPU) orapplication processor (AP)) that can perform the corresponding actionsby executing one or more software programs stored in the memory device.

In this disclosure, the term user may refer to a person who uses anelectronic apparatus or an apparatus (e.g., artificial intelligenceelectronic apparatus) that uses an electronic apparatus.

Hereinafter, the embodiments will be described in a greater detail withreference to the drawings.

FIGS. 1 and 2 are flow charts to describe a method for providing amanual of an electronic apparatus according to various embodiments ofthe disclosure.

Referring to FIGS. 1 and 2, a method for providing a manual may providea display device with one of a general manual, a detailed manual, and asimple manual for an external device in operations S110 and S210.

The user response to the provided manual can be obtained in operationsS120 and S220. The user response indicates the feedback information ofthe user who viewed or listened to the provided manual. The userresponse may include user voice information or image information for theprovided manual. The user voice information may include voiceinformation uttered by a user listening to or viewing the providedmanual, and the user image information may include facial expressioninformation of the user or gesture information (or pose information) ofthe user, such as a user listening to or viewing the provided manual.

If a user response is not obtained in operation S220-N, user responseinformation can be requested to the user terminal device in operationS230. In this case, the user terminal device receiving the user responserequest information can provide voice information requesting the userresponse to the user in operation S240. The user terminal device may bea display device that is providing a manual, but this is merelyexemplary. The user terminal device may be an electronic apparatus suchas a cellular phone, a smart phone, a TV, a monitor, an air conditioner,a refrigerator, and a washing machine. For example, the user terminaldevice, based on the received user response request information, mayprovide the user with a voice such as “how do you feel about themanual?” “did you do it well as the manual?”. In response, the userterminal device may receive response information from the user. Forexample, the user terminal device can obtain voice information “no, itis difficult” from a user, or capture a facial expression or gesture ofthe user. The user terminal device receiving the user response requestinformation may capture the user and transmit the captured imageinformation to the electronic apparatus.

When the electronic apparatus obtains the user response information inoperation S240-Y, the electronic apparatus may determine theunderstanding level for the manual provided based on the obtained userresponse in operation S250. When the electronic apparatus does notobtain the user response information in operation S240-N, the electronicapparatus may request the user response information to the user terminaldevice again in operation S230.

Then, the understanding level for a manual provided on the basis of theuser response can be determined in operations S130 and S250. Theunderstanding level refers to the degree of understanding of the userwith respect to the manual, which indicates the degree to which the userrecognizes and accepts the content of the manual. The understandinglevel indicates whether more or less information is recognized than theinformation contained in the content of the manual provided to the user.The understanding level can be divided into a high level, a neutrallevel, and a low level.

The high level means the understanding of the user for the manual ishigh, and indicates that the user is identified to recognize moreinformation than the information contained in the content displayed onthe display device. For example, if a step A out of steps A, B, and C ina manual is being provided to the display device, when the user responseof “I completed step B” is received, the user's understanding level canbe determined to be a high level. If the user's understanding level ishigh level, the manual provided to the display device can be substitutedwith another manual. At this time, the substitute manual may include asmall amount of information summarizing the manual information providedto the display device, and the user may obtain the manual contentinformation within a time faster than the previously provided manual.

The neutral level represents that an understanding level of the user forthe manual is neutral, and indicates the understanding level when it isdetermined that the user recognizes information corresponding to theinformation included in the content displayed on the display device. Forexample, if a step A out of steps A, B, C included in the manual isprovided to the display device, and the user response of “I completedstep A” is received, the user's understanding can be determined to be aneutral level. If the user's understanding level is a neutral level, themanual provided to the display device may continue to be provided to theuser.

Finally, the low level means that the understanding level of the userfor the manual is low level and indicates the user's understanding whenit is determined that the user recognizes less information than theinformation contained in the content displayed on the display device.For example, if a step A out of steps A, B, and C of the manual isprovided to the display device, and the user response “I don't know” isreceived, the user's understanding level can be determined to be a lowlevel. If the user's level of understanding is low, the manual providedto the display device can be substituted with another manual. In thiscase, the substitute manual may include a large amount of information indetail in the manual information provided to the display device.

Based on the obtained understanding level, a determination can be madeas to whether substitution of the manual is necessary in operation S260.Specifically, if the determined understanding is at a low level or ahigh level, it may be determined that substitution of the manual isnecessary. In addition, if the determined understanding level is aneutral level, a manual that is identical with the manual provided tothe display device may be provided without substitution of the manual.However, this is merely exemplary, and if the determined understandinglevel is a neutral level, it may be determined that substitution of themanual is necessary.

In accordance with the understanding level, the provided manual may besubstituted with one of the general manual, detailed manual, and asimple manual in operation S140.

According to the determined understanding level, a general manual,detailed manual, and simple manual may be selected as a substitutemanual in operation S270. Prior to transmitting the substitute manual,the substitute manual may be selected in consideration of the manualtransmitted to the display device. A specific description will bedescribed with respect to FIG. 4.

The controlling method may determine the reproduction starting point ofthe selected manual in operation S280. The reproduction starting pointof the manual selected based on the state information of an externaldevice may be determined. A specific description will be given withrespect to FIG. 6.

If the content provided to the display device is a general manual, andthe understanding level is determined to be a high level, the substitutemanual may be determined to be a simple manual. Conversely, if thecontent provided to the display device is a general manual, and theunderstanding level is determined to be a low level, the substitutemanual may be determined to be the detailed manual.

The provided manual may be substituted with a selected manual inoperation S290. In this case, the selected manual and the information onthe reproduction starting point may be transmitted to the displaydevice, to substitute the manual provided to the display device. To bespecific, the address information corresponding to the selected manualand the information on the reproduction starting point may be providedto the display device to substitute the provided manual.

As another embodiment, the selected manual may be transmitted to thedisplay device by streaming. In this case, the data from the determinedreproduction starting point among the entire reproduction section of theselected manual may be streamed on the display device.

FIG. 3 is a view to describe a method for providing a manual accordingto an embodiment of the disclosure. Specifically, FIG. 3 is a view todescribe S250 of FIG. 2 in greater detail.

Referring to FIG. 3, after the user response for the provided manual isobtained in operation S220, semantic information of the voiceinformation and height information of the voice information included inthe user response may be obtained in operation S310.

By using the semantic information and height information of the obtainedvoice information, the understanding level may be determined.

When it is determined that the semantic information of the user voiceinformation is positive in operation S320, it may be determined that theunderstanding level is a high level in operation S330. However, if it isdetermined that the semantic information of the user voice informationis not positive in operation S320, it may be determined whether thesemantic information of the user voice information corresponds tonegative in operation S340.

When it is determined that the semantic information of the user voicecorresponds to negative in operation S340-Y, it may be determined thatthe understanding level is a low level in operation S350. Conversely, ifit is determined that the semantic information of the user voiceinformation does not correspond to negative in operation S340-N, thatis, the semantic information of the user voice information does notcorrespond to positive and negative, the understanding level may bedetermined based on the voice height information of the user voiceinformation.

When the information of the voice height of the user voice informationis greater than or equal to a predetermined first value in operationS360, it may be determined that the understanding level is the highlevel. Conversely, when the information on the voice height of the uservoice information is less than a predetermined first value in operationS360, it may be determined that the voice height information of the uservoice information is less than a predetermined second value in operationS370.

When the voice height information of the user voice information is lessthan a predetermined second value in operation S370, it may bedetermined that the understanding level is a low level in operationS350. Conversely, when the voice height information of the user voiceinformation is greater than or equal to a predetermined second value, itmay be determined that the understanding level is a neutral level inoperation S380.

FIGS. 4 and 5 are views to specifically describe the step S270 of FIG. 2according to various embodiments of the disclosure. When it isdetermined that substitution of the manual provided to the displaydevice is deemed necessary based on the understanding level determinedby the method of FIG. 3, the figures describe a method for selecting amanual to be substituted.

Referring to FIG. 4, the method for providing a manual may determinewhether the determined understanding level corresponds to a high levelin operation S410. When the determined understanding level correspondsto the high level in operation S410-Y, the manual provided to thedisplay device, that is, the manual reproduced in the display device maybe determined in operation S420.

It may be determined whether the manual provided to the display devicecorresponds to the general manual in operation S430. When the manualprovided to the display device corresponds to the general manual inoperation S430-Y, the simple manual may be selected as the substitutemanual in operation S440.

If the manual provided to the display device is not a general manual, itmay be determined whether the provided manual corresponds to a simplemanual in operation S450. If the manual provided to the display devicecorresponds to the simple manual (i.e., if the understanding level ofthe manual corresponds to the high level and the manual reproduced onthe display device corresponds to a simple manual), the simple manualthat has been already provided becomes a substitute manual and, aseparate manual may not be selected in operation S450-Y. If the providedmanual does not correspond to a simple manual in operation S450-N, it ispossible to determine whether the provided manual corresponds to adetailed manual in operation S460. In this case, if the provided manualcorresponds to a detailed manual in operation S470, a general manual ora simple manual can be selected in operation S470.

In operation S410, if the understanding level does not correspond to thehigh level, the operations will be described with reference to FIG. 5.

Referring to FIG. 5, when the understanding level does not correspond tothe high level, it may be determined whether the understanding levelcorresponds to the low level in operation S510.

When the understanding level corresponds to the low level, the manualprovided to the display device may be determined in operation S520.

In operation S530, it is possible to determine whether the manualprovided to the display device corresponds to a general manual. If themanual provided to the display device corresponds to the general manualin operation S530-Y (i.e., when the understanding level is a low leveland the manual provided to the display device corresponds to a generalmanual), a detailed manual can be selected in operation S540. If themanual provided to the display device is not a general manual inoperation S530-N, it is possible to determine whether the manualprovided to the display device is a simple manual in operation S550. Ifthe manual provided to the display device is a simple manual inoperation S550-Y (i.e., when the understanding level is low level andthe manual provided to the display device is a simple manual), a generalmanual or a detailed manual can be selected in operation S560. If themanual provided to the display device is not a simple manual inoperation S550-N (i.e., if the manual provided to the display device isa detailed manual), the detailed manual, which has been alreadyprovided, becomes a substitute manual. Thus, the detailed manual whichhas been provided may become a substitute manual without selecting aseparate manual in operation S550.

FIG. 6 is a view to specifically describe step S280 of FIG. 2 accordingto an embodiment of the disclosure.

Referring to FIG. 6, the method for providing a manual may includedetermining whether state information of the external device is changedin operation S610. For this purpose, state information of the externaldevice may be received from the external device. The external device isan electronic apparatus described in the manual, and may be anelectronic apparatus included in a content of the manual.

When a manual corresponding to the external device is displayed in thedisplay device, the external device can determine the state of theexternal device and transmit the determined state information of theexternal device to the electronic apparatus. The state information ofthe external device refers to the information of the components(hardware) or software of the external device necessary to perform thefunctions of the external device. For example, if the external device isan air conditioner, whether power is connected, whether a filter isprovided, air vent state information, outdoor unit temperature, pairingwith a user terminal device, or the like, can be state information ofthe air conditioner.

When it is determined that the state information of the external deviceis changed after the manual is transmitted to the display device inoperation S610, a reproduction section, among a plurality of sections ofthe selected substitute manual, corresponding to the next reproductionsection of the section reproduced in the display device may bedetermined as a reproduction starting point in operation S620.

In addition, in order for the next section of the section reproduced inthe display device to be displayed on the display device, the nextsection of the section reproduced by the display device among theplurality of sections is set as a reproduction section, and thereproduction starting point of the set reproduction section can bedetermined as the starting point of reproduction of the substitutemanual.

If it is determined that the state information of the external device isnot changed in operation S610, it is possible to set a reproductionstarting point corresponding to the reproduction section in the displaydevice among the plurality of sections of the selected substitute manualin operation S630. That the state information of the external device isnot changed means that the state of the external device is not changedso as to correspond to the section reproduced in the display device. Inthis regard, the already reproduced section may be set as a reproductionsection of the substitute manual so that the selected substitute manualreproduces the section reproduced in the display device again, and setthe starting point of the set reproduction section as the starting pointof the substitute manual.

As described above, information on section of the manual reproduced inthe display device is necessary to determine the reproduction startingpoint and the electronic apparatus may receive information on thesection information of the reproduced manual from the display device.

FIG. 6 illustrates a method of determining a reproduction point of theselected manual based on the state information of the external deviceafter the step S270 of FIG. 2, i.e. after selecting one of a generalmanual, a detailed manual, and a simple manual on the basis of theunderstanding level. In some cases, it is possible to determine thereproduction point of the substitute manual in step S270, which replacesthe manual based on the understanding level of the user. For example, ifthe manual which has been reproduced on the display device includes aplurality of sub-content, the step of selecting the substitute manualmay be the step of selecting one of the plurality of sub-contentincluded in the substitute manual. In addition, in that the plurality ofsub-content included in the substitute manual is sequentiallyreproduced, the step of selecting one of the plurality of sub-contentsincluded in the substitute manual can be considered to determine thereproduction point of the manual.

FIG. 7 is a view to describe a manual providing system for providing amanual according to a user's understanding level according to anembodiment of the disclosure.

Referring to FIG. 7, a manual providing system 1000 may include theelectronic apparatus 100, the display device 200, and a user terminaldevice 300.

The electronic apparatus 100 may provide a manual including variouscontents to the display device 200. In the disclosure, the content meansa character, a figure, a color, a drawing, sound, voice, or combinationthereof, and means information provided as a text, an image, a video,voice, or a combination thereof.

The electronic apparatus 100 may provide a manual stored in theelectronic apparatus 100 to the display device 200, and may provide themanual received from an external device (not shown) to the displaydevice 200. In addition, although the electronic apparatus 100 maydirectly transmit the manual to the display device 200, the electronicapparatus 100 may transmit the location information (for example,uniform resource locator (URL) information) stored in the manual to thedisplay device 200.

The display device 200 may provide a manual received from the electronicapparatus 100 to a user.

The display device 200 may be various user terminal devices including adisplay and may be a notebook, a mobile phone, a smartphone, a tabletpersonal computer (PC), a kiosk, or the like, as well as a television(TV) and a monitor.

The user terminal device 300 may obtain the user's voice information orimage information about the manual provided by the display device 200.The user's voice information is information on the sound generated bythe user who received the manual provided by the display device 200 andcan include a non-verbal sound, such as a sigh, cheer, admiration aswell as a verbal sound, such as a user utterance. In addition, the imageinformation of the user may include a user's facial expression, gesture,or pose.

The user terminal device 300 may transmit the obtained user voiceinformation or image information to the electronic apparatus 100. Forthis purpose, the user terminal device 300 may include a microphone or acamera.

Referring to FIG. 7, the user terminal device 300 is illustrated as aspeaker, but is not limited thereto, and a device including a microphoneor a camera may be the user terminal device 300 of FIG. 7.

The electronic apparatus 100 may determine whether the manual providedby the display device 200 needs to be substituted based on the uservoice information or image information received from the user terminaldevice 300.

The electronic apparatus 100 may determine the understanding level ofthe user about the manual from the user voice information or the imageinformation, and determine whether the corresponding manual issubstituted based on the determined understanding level. Based on uservoice information or image information, the electronic apparatus 100 candetermine that the manual provided to the user should be substituted ifthe user's understanding level of the manual is lower or higher than thepreset range.

When it is determined that the manual provided to the user needs to besubstituted, the electronic apparatus 100 may provide a content that isdifferent from the manual provided to the user based on the determinedunderstanding level.

According to another embodiment of the disclosure, the display device200 and the user terminal device 300 may be implemented as a singledevice. The display device 200 may obtain the user's voice informationor image information for the content provided by the display device 200,and provide the information to the electronic apparatus 100. In thiscase, the electronic apparatus 100 may receive user voice informationfrom a remote controller (not shown) for controlling the display device200, as well as the display device 200.

The electronic apparatus 100 may determine whether the content providedby the display device 200 is suitable to a user through voiceinformation or image information of a user received from the displaydevice 200 or a remote controller (not shown) for controlling thedisplay device.

FIG. 8 is a block diagram illustrating a configuration of an electronicapparatus 100 according to an embodiment of the disclosure.

Referring to FIG. 8, the electronic apparatus 100 includes acommunication interface 110, a memory 120, and a processor 130.

The communication interface 110 is a component for the electronicapparatus 100 to communicate with the display device 200 and the userterminal device 300. Through the communication interface 110, theelectronic apparatus 100 may provide content or information aboutcontent to the display device 200 and may receive user voice informationor image information from the user terminal device 300.

The communication interface 110 may perform communication with anotherexternal device (not shown as well as the display device 200 and theuser terminal device 300. The communication interface 110 may receivestate information of an external device (not shown) throughcommunication with an external device (not shown).

The communication interface 110 may include various communicationmodules such as a wired communication module (not shown), near-fieldwireless communication module (not shown), wireless communication module(not shown), or the like.

The wired communication module is a module for performing communicationwith an external device (not shown) according to a wired communicationmethod such as wired Ethernet. The near field communication module is amodule for performing communication with an external terminal (notillustrated) positioned at a near distance through near fieldcommunication methods such as Bluetooth (BT), Bluetooth low energy(BLE), ZigBee, or the like. The wireless communication module is amodule which communicates by being connected to an external networkaccording to a wireless communication protocol, such as wirelessfidelity (Wi-Fi), Institute of Electrical and Electronics Engineers(IEEE) and the like. The wireless communication module may furtherinclude a mobile communication module which is connected to a mobilecommunication network according to various mobile communicationstandards such as 3rd generation (3G), 3rd generation partnershipproject (3GPP), long term evolution (LTE), LTE advanced (LTE-A), 5thgeneration (5G) networks, or the like, to perform communications.

The memory 120 is a component for storing various programs, data, or thelike, that are necessary for operating the electronic apparatus 100. Thememory 120 may be implemented as a non-volatile memory, a volatilememory, a flash memory, a hard disk drive (HDD), a solid state drive(SSD), or the like. The memory 120 is accessed by the processor 130 andreading/writing/modifying/deleting/updating of data by the processor 130may be performed. In the disclosure, the term memory may include thememory 120, read-only memory (ROM) in the processor 130, random accessmemory (RAM), or a memory card (for example, a micro secure digital (SD)card, and a memory stick) mounted to the electronic apparatus 100.

The memory 120 may store various content provided to the display device200 or information on the content. The information on the contents mayinclude address information of the content where the content is stored,content reproduction history information, content section information,or the like.

The memory 120 may include an artificial intelligence (AI) agent. The AIagent is a dedicated program to provide AI-based services (e.g., voicerecognition services, secretarial services, translation services, searchservices, etc.) and may be executed by existing general-purposeprocessors (e.g., a central processing unit (CPU) or separate AI-onlyprocessors (e.g., a graphics processing unit (GPU)). When the AI agentis executed by the aforementioned processor, the AI agent may analyzeuser voice information or image information, obtain the user's emotionor understanding level, and provide a response to the obtained emotionor understanding level.

For this purpose, the memory 120 may include at least one configuration(or module) for obtaining the user's emotion through information on theuser's expression, degree of attitude, or the like, included in theuser's image.

The memory 120 may include a plurality of configurations (or modules)constituting the dialogue system. Here, the dialogue system is a programfor performing voice recognition for the user's voice and providing aresponse thereto. The dialogue system will be further described withreference to FIG. 9.

FIG. 9 is a block diagram illustrating a dialogue system of anelectronic apparatus according to an embodiment of the disclosure.

Referring to FIG. 9, a dialogue system 400 is a configuration forperforming a dialogue through a natural language with a user. Thedialogue system 400 may be stored in the memory 120 of the electronicapparatus 100 according to an embodiment. However, this is merelyexemplary, and at least one configuration included in the dialoguesystem 400 may be included in an external device (e.g., the userterminal device 300, another server (not shown)). Some configurations ofthe dialogue system 400 may be included in the user terminal device 300.

Referring to FIG. 9, the dialogue system 400 may further include anautomatic speech recognition (ASR) module 410, a natural languageunderstanding (NLU) module 420, a dialogue manager (DM) module 430, anatural language generator (NLG) module 440, and a text to speech (TTS)module 450. The dialogue system 400 may further include a knowledge base(not shown).

The ASR module 410 may convert user input received from the userterminal device 300 into text data. For example, the ASR module 410 mayinclude a speech recognition module. The speech recognition module mayinclude an acoustic model and a language model. For example, theacoustic model may include information related to speech, and thelanguage model may include information on unit phoneme information and acombination of unit phoneme information. The speech recognition modulemay convert the user utterance into text data using the informationrelated to speech and information on the unit phoneme information.Information about the acoustic model and language model may be storedin, for example, an automatic speech recognition database (ASR DB).

The natural language understanding module 420 may recognize theintention of a user by performing syntactic analysis or semanticanalysis. Grammatical analysis may divide the user input in grammaticalunits (e.g., words, phrases, morphemes, or the like), and grasp whichgrammatical elements the divided units have. The semantic analysis maybe performed using semantic matching, rule matching, formula matching,or the like. Accordingly, the natural language understanding module 420may acquire domain or intent for the user input.

The natural language understanding module 420 may determine userintention and parameters using the matching rule divided into a domainand an intention. For example, the natural language understanding module420 may include the one domain (e.g., air conditioner, TV, or the like)may include a plurality of intents (e.g., manual searching, manualreproduction, or the like), and one intention may include a plurality ofparameters (e.g., search target, reproducing device, or the like). Thematching rules may be stored in a natural language understandingdatabase (NLU DB).

The natural language understanding module 420 may grasp the meaning of aword extracted from a user input using a linguistic characteristic(e.g., a grammatical element) such as a morpheme or a phrase, anddetermine a user intention by matching the grasped meaning with thedomain and the intention. For example, the natural languageunderstanding module 420 may determine the user's intention bycalculating how many words extracted from user input are included ineach domain and intention. The natural language understanding module 420may determine the parameters of the user input using words that become abasis for understanding the intent. The natural language understandingmodule 420 may determine the user's intention using the natural languagerecognition database (not shown) in which the linguistic characteristicfor grasping the intention of the user input is stored.

A dialogue manager module 430 may determine whether the intention of auser grasped by the natural language understanding module 420 is clear.For example, the dialogue manager module 430 may determine whether theintention of the user is clear based on whether the parameterinformation is sufficient. The dialogue manager module 430 may determinewhether the parameter grasped by the natural language understandingmodule 420 is sufficient to determine the user's understanding level ofthe content. The dialogue manager module 430 may perform feedback torequest necessary information to the user if the user intension is notclear. For example, the dialogue manager module 430 may perform thefeedback to request information about parameters for grasping the userintention. In addition, the dialogue manager module 430 may generate andoutput a message for checking a user inquiry including the text changedby the natural language understanding module 420.

The dialogue manager module 430 may include a content provider module(not shown). The content provider module may determine the understandinglevel based on the parameter and the intention grasped by the naturallanguage understanding module 420 and generate information correspondingto the determined understanding level.

The dialogue manager module 430 may provide response information for theuser voice using a knowledge base (not shown). The knowledge base isdatabase in which various knowledge information are stored in a form ofontology such as a resource description framework (RDF), web ontologylanguage (OWL), or the like.

The natural language generation module (NLG module) 440 may change thedesignated information into a text form. The information changed in thetext form may be a form of natural language utterance. The designatedinformation may be, for example, information about an additional input,information for guiding completion of an action corresponding to a userinput, or information for guiding an additional input of a user (forexample: feedback information for a user input). The information changedin the text form may be displayed on a display (not shown) or changedinto a voice form by a text-to-speech (TTS) module 450.

The TTS module 450 may change the information of the text format tovoice format information. The TTS module 450 may receive information ofa text format from the natural language generation module 440, changethe information of the text format into information of a voice format,and output the same through a speaker.

The natural language understanding module 420 and the dialogue managermodule 430 may be implemented as one module. For example, the naturallanguage understanding module 420 and the dialogue manager module 430may be implemented as one module to determine the intention of the userand the parameter, and obtain a response corresponding to the determineduser intention and the parameter.

The memory 120 may store content. The content may include a plurality ofmanuals for an external device (not shown). The manual for an externaldevice (not shown) represents content including information necessaryfor a user to use, maintain, update or repair an external device, suchas hardware, software, and system included in an external device (notshown) or an external device (not shown), a method of use, a problemsolving method, and the like. The manual may be provided in the form oftext, images, or voice, and may be provided in the form of combiningsome of the text, images, and voice.

A plurality of manuals for an external device (not shown) may include ageneral manual, a detailed manual, and a simple manual. The memory 120may include the detailed manual and the simple manual corresponding tothe general manual.

FIG. 10 is a view to describe a plurality of manuals according to anembodiment of the disclosure.

Referring to FIG. 10, a simple manual 1010, a general manual 1020, and adetailed manual 1030 are disclosed.

While the general manual 1020 may include content corresponding to aplurality of steps (step 1, step 2, and step 3) necessary for using,maintaining, updating, or repairing an external device (not shown), thesimple manual 1010 and the detailed manual 1030 may include a contentcorresponding to a plurality of steps included in the general manual1020.

The simple manual 1010 may include a smaller amount of soundinformation, text information, and image information than the generalmanual 1020 for at least one of a plurality of steps of step 1, step 2,and step 3 included in the simple manual 1010. The detailed manual 1030may include a greater amount of sound information, text information, andimage information than the general manual 1020 for at least one of aplurality of steps of step 1, step 2, and step 3 included in thedetailed manual 1030.

Accordingly, the play time of at least one of the plurality of steps(step 1, step 2, and step 3) included in the simple manual 1010 can beshorter than the reproduction time of the corresponding step of thegeneral manual 1020. In addition, the reproduction time of at least oneof the plurality of steps (step 1, step 2, and step 3) included in thedetailed manual 1030 may be shorter than the reproduction time of thecorresponding step of the general manual 1020.

The simple manual 1010 or the detailed manual 1030 includes a pluralityof steps of the general manual 1020, and thus the simple manual 1010 orthe detailed manual 1030 may include points 1011 and 1013 correspondingto a particular point 1011 in the general manual 1020. At this time, thereproduction time of the particular point 1011 of the simple manual 1010can be faster than the reproduction time of the particular point 1012 ofthe general manual 1020, and the reproduction time of the particularpoint 1013 of the detailed manual 1030 can be later than thereproduction time of the particular point 1012 of the general manual1020.

The detailed manual 1020 may be a manual including an image captured atan angle other than the general manual 1010. For example, the detailedmanual may include images captured at various angles of the externaldevice, such as the back, side, top, bottom, etc. of an external device(not shown). In addition, the detailed manual 1020 may include an imagethat magnifies an image included in the general manual 1010.

The detailed manual 1020 may provide a user with more information,various information (for example, an image at another angle), detailedinformation (for example, a magnified image) than the general manual1010.

The general manual 1020, the simple manual 1010, and the detailed manual1030 are not always stored in the memory 120. The general manual 1020,the simple manual 1010, and the detailed manual 1030 are stored in anexternal electronic apparatus such as a content server (not shown), andaddress information of the general manual 1020, the simple manual 1010,and the detailed manual 1030 stored in the external electronic apparatusmay be stored in the memory 120.

Referring back to FIG. 8, the processor 130 may be electricallyconnected to the memory 120 to control the overall operations andfunctions of the electronic apparatus 100. For example, the processor130 may drive an operating system or application program to controlhardware or software components connected to the processor 130, and mayperform various data processing and operations. The processor 130 mayalso load and process commands or data received from at least one of theother components into volatile memory and store the various data innon-volatile memory.

For this purpose, the processor 130 may execute one or more softwareprograms stored in a dedicated processor (e.g., embedded processor) forperforming the operations or memory device and may be implemented as ageneral-purpose processor (for example: central processing unit (CPU) orapplication processor) capable of performing the correspondingoperations.

In order to control the operations and functions of the electronicapparatus 100, the processor 130 may include an understandingdetermination module (or configuration) 131, a manual determinationmodule (or configuration) 132, and a manual reproduction pointdetermination module (or configuration) 133. Although the understandingdetermination module 131, the manual determination module 132, and themanual reproduction point determination module 133 may each be aseparate configuration, at least two of the modules (or configurations)may be combined into a single module (or configuration).

The processor 130 may control the communication interface 110 to providea content to the display device 200.

The processor 130 may control the communication interface 110 to provideone of a plurality of manuals for an external device to the displaydevice 200. The plurality of manuals may include the general manual, thedetailed manual, and the simple manual.

The processor 130 may obtain a user response to the provided manual. Theprocessor 130 may communicate with the user terminal device 300 or thedisplay device 200 to receive a user response to the manual. The userresponse may include user voice information or image information. Theuser voice information may include voice information corresponding tothe user's utterance, and the user image information may include theuser's facial expression information or the user's attitude information,or the like.

The processor 130 may determine a user's understanding of the manualbased on the obtained user response. An understanding level indicates adegree of user's understanding of a content, and the understanding levelmay be divided into a high level, a neutral level, and a low levelaccording to the degree of understanding. It is to be understood thatthis is merely exemplary, and may be divided into various stepsdepending on the user's settings or systems. Determination on theunderstanding level based on user response will be described in greaterdetail below in FIG. 12.

The processor 130 may use the state information of the external deviceto determine an understanding level of the manual. The external devicerepresents an electronic apparatus included in the manual, and morespecifically, the external device can be an electronic apparatusdescribed in the content included in the manual. The state informationof the external device indicates information related to the status ofthe external device while the manual is displayed on the display device200. The state information of the external device refers to the stateinformation of a component (hardware) or software of the external devicenecessary to perform the function of the external device. For example,when the external device is an air conditioner, whether power isconnected, whether a filter is provided, air vent state information,outdoor temperature information, pairing with a user terminal device, orthe like, can be state information of the air conditioner.

For this purpose, the processor 130 may receive state information of theexternal device from an external device (not shown). The stateinformation of the external device may be generated at an externaldevice, and the processor 130 may request state information of theexternal device to an external device (not shown). However, this is onlyone embodiment, and an external device (not shown) may periodicallytransmit state information of the external device to the electronicapparatus 100. According to another embodiment of the disclosure, if theprocessor 130 provides a manual to the display device 200, the processor130 may request state information of the external device to an externaldevice included in the manual provided to the display device 200, and anexternal device (not shown) receiving the request of the stateinformation from the processor 130 may generate the state information ofthe external device and transmit the information to the processor 130.

The processor 130 may receive the state information of an externaldevice from an external device that is a subject of the manual in orderto determine an understanding level. For example, when the manual isrelated to an air-conditioner, the processor 130 may receive the stateinformation of the air-conditioner.

The processor 130 may determine an understanding level based on thereceived state information of the external device. For example, if thestate information of the external device is not changed for apredetermined period of time after providing the manual to the displaydevice 200, the processor 130 may determine an understanding level to alow level. In addition, if the state information of the external deviceis changed for a predetermined time after providing the manual to thedisplay device 200, the understanding level of the state of the externaldevice can be determined to be a neutral level or a high level accordingto the degree of change of the state of the external device.

Regarding this, the processor 130 may determine whether it is necessaryto substitute the manual provided to the display device 200 with anothermanual based on the determined understanding level. For example, whenthe understanding level is determined to be a low level or a high level,the processor 130 may determine that it is necessary to substitute themanual.

The processor 130 may determine a manual to be substituted (hereinafter,referred to as a substitute manual) by using a manual determinationmodule 132. To be specific, the processor 130 may determine thesubstitute manual based on the manual provided to the display device 200and the user's understanding level of the provided manual. In this case,the substitute manual may be one of a general manual, a detailed manual,and a simple manual.

If the user's understanding level is a high level and the manualreproduced by the display device 200 is not a simple manual, theprocessor 130 can determine the manual including information of ashorter reproduction time or a smaller amount of information than themanual provided by the display device 200 as the substitute manual. Forexample, if the user's understanding level is a high level and themanual reproduced on the display device is a general manual, theprocessor 130 may determine the simple manual as a substitute manual. Inaddition, if the user's understanding level is a high level and themanual reproduced in the display device is a detailed manual, theprocessor 130 can determine the general manual or the manual as asubstitute manual.

If the user's understanding level is a low level and the manual providedto the display device 200 is a detailed manual, the processor 130 candetermine the manual including a greater reproduction time or a greateramount of information than the manual provided by the current displaydevice 200 as a substitute manual. For example, if the user'sunderstanding level is a low level and the manual reproduced on thedisplay device is a general manual, the processor 130 may determine thedetailed manual as a substitute manual. In addition, if the user'sunderstanding level is a low level and the manual reproduced in thedisplay device is a simple manual, the processor 130 can determine thegeneral manual or the detailed manual as a substitute manual.

The processor 130 may determine the reproduction section or reproductionstarting point of the determined substitute manual. The processor 130may use the manual reproduction point determination module 133.

The processor 130 may determine the reproduction starting point of thesubstitute manual based on the voice recognition result for the uservoice information. If the voice information included in the userresponse includes the state information of the external device, theprocessor 130 can determine the state information of the external devicebased on the recognition result of the user voice information, and candetermine the next section of the section corresponding to the stateinformation of the external device during the reproduction section ofthe substitute manual as the reproduction starting point. For example,if the user utters “I removed an air conditioner filter,” the processor130 can determine that the state information of the external device isin a state that the air conditioner filter has been removed based on theresult of the user voice recognition. In addition, the processor 130 candetermine the next section of the section corresponding to the airconditioner filter removal step as the reproduction starting point. Forthis, the memory 120 may store time information for each sectionincluded in the manual, state information of an external devicecorresponding to each section, or the like.

In another embodiment of the disclosure, the processor 130 may determinea reproduction starting point of the substitute manual based on anunderstanding level obtained according to a voice recognition result forthe user voice information. When the understanding level of the user isa low level, the processor 130 may set a section corresponding to thelast reproduced section of the display device 200 to the reproductionsection of the substitute manual, and determine the starting point ofthe set reproduction section as the reproduction starting point of thesubstitute manual. If the understanding level of the user is the neutrallevel or the high level, the processor 130 may set a sectioncorresponding to the next section of the last reproduced section in thedisplay device 200 to the reproduction section of the substitute manual,and determine the starting point of the set reproduction section as thereproduction starting point of the substitute manual.

The processor 130 may determine the reproduction starting point in thesubstitute based on the state information received from the externaldevice.

If it is determined that the state information of the external device ischanged after receiving the state information of the external device,and transmitting the manual to the display device 200, the processor 130may set a section corresponding to the next section of the sectionreproduced by the display device during the plurality of sections of thesubstitute manual as the reproduction section. In addition, theprocessor 130 may determine a reproduction starting point of the setreproduction section as a reproduction starting point of the substitutemanual. Accordingly, the display device 200 may provide the nextreproduction section of the previously reproduced section to the user.

It may be determined that the state information of the external deviceis not changed after the manual is transmitted to the display device200. That the state information of the external device is not changedmeans the state of the external device is not changed to correspond tothe section reproduced in the display device and thus, the displaydevice 200 needs to reproduce the section of the previously reproducedin the manual again. Accordingly, if it is determined that the manual istransmitted to the display device 200 and the state information of theexternal device is not changed, the processor 130 may set a sectioncorresponding to the section reproduced by the display device among theplurality of sections of the substitute manual as the reproductionsection. In addition, the processor 130 may determine a reproductionstarting point of the set reproduction section as a reproductionstarting point of the substitute manual.

As such, when the reproduction starting point is determined based on thestate information of the external device, in that information on thesection of the manual reproduced in the display device is necessary, theelectronic apparatus 100 may receive information on the sectioninformation of the reproduced manual from the display device.

The processor 130 may determine a reproduction starting point in thesubstitute manual based on the content reproduction history information.For example, the processor 130 may skip a section including informationreproduced for more than the predetermined number of times and determinea reproduction starting point of the substitute manual in considerationof the user's content reproduction history information.

The processor 130 may transmit the information on the reproductionstarting point and the substitute manual to the display device 200 andsubstitute the manual provided to the display device 200. The processor130 may provide the address information corresponding to the substitutemanual and the information on the reproduction starting point to thedisplay device 200 and substitute the provided manual.

In another embodiment of the disclosure, the processor 130 may transmita manual to the display device 200 in a streaming manner. In this case,the processor 130 may stream the data from the determined reproductionstarting point among the entire reproduction sections of the substitutemanual to the display device 200.

It has been described that the simple manual 710, the general manual720, and the detailed manual 730 including a plurality of steps (step 1,step 2, and step 3) are stored in the memory 120, but it is not limitedthereto.

Each of the general manual, detailed manual, and the simple manual maybe divided by a plurality of sub-contents sequentially reproduced, andthe memory 120 may store a plurality of sub-contents.

FIG. 11 is a view to describe a display device for selecting asub-content to be reproduced among a plurality of sub-contents relatedto an external device according to an embodiment of the disclosure.

Referring to FIG. 11, it is assumed that a plurality of steps (step 1,step 2, . . . , step N) are necessary to use, maintain, update or repairthe external device. In this case, a general manual, a detailed manual,and a simple manual can be separately stored in the memory 120 accordingto a plurality of steps. For example, in the memory 120, the simplemanual may be divided into a plurality of sub-content 1111, 1112, and1113 according to a plurality of steps, and may be stored, the generalmanual may be divided into a plurality of sub-content 1121, 1122, and1123 according to a plurality of steps, and stored, and the detailedmanual may be divided into a plurality of sub-content 1131, 1132, and1133 according to a plurality of steps and stored.

The processor 130 may transmit one of a plurality of sub-contents 1111,1112, 1113, 1121, 1122, 1123, 1131, 1132, and 1133 stored in the memory120 to the display device 200.

While the transmitted sub-content is being reproduced in the displaydevice, or after the transmitted sub-content is reproduced, theprocessor 130 may receive a user response for the reproduced sub-contentfrom the user terminal device 300.

As described in FIG. 8 above, the processor 130 may determine a user'sunderstanding level of the reproduced sub-content based on the receiveduser response, and may determine a substitute manual based on theunderstanding level. For example, if the content provided to the displaydevice 200 is a general manual and the understanding level is determinedto be a high level, the processor 130 may determine the alternate manualas a simple manual. Conversely, if the content provided to the displaydevice 200 is a general manual and the understanding level is determinedto be a low level, the processor 130 can determine the substitute manualas a detailed manual.

The processor 130 may select one of a plurality of contents 1111, 1112,1113, 1121, 1122, 1123, 1131, 1132, and 1133 based on the sub-contentprovided to the display device 200 and the determined understandinglevel.

The processor 130 may select sub-content corresponding to the next stepof sub-content provided to the display device 200 among the plurality ofsub-contents of the substitute manual when the understanding level ishigh, and select sub-content corresponding to the step of thesub-content provided to the display device 200 among the plurality ofsub-content of the substitute manual when the understanding level is lowlevel. For example, if sub-content 1121 for step 1 is provided todisplay device 200, and the understanding level of sub-content 1121 isdetermined to be high, the processor 130 may select content 1112corresponding to step 2, which is the next step of step 1 among theplurality of sub-content 1111, 1112, and 1113 of the simple manual.Conversely, if it is determined that the understanding level of thesub-content 1121 is low, the processor 130 may select the content 1131corresponding to the step 1 among the plurality of sub-contents 1131,1132, and 1133 of the detailed manual.

The processor 130 may provide the selected sub-content to the displaydevice 200.

According to the descriptions of FIGS. 8 and 11, the electronicapparatus 100 may reflect the user's understanding level of the manualprovided to the display device 200 and provide a new manual to a user,and the user may receive a new manual to correspond to the user'sunderstanding level for the provided manual.

FIG. 12 is a view to describe an electronic apparatus for determining anunderstanding level from user voice information according to anembodiment of the disclosure.

Referring to FIG. 12, the processor 130 may receive user voiceinformation as the user response to the provided content. The processor130 may obtain emotion parameter from the received user voiceinformation.

The emotion parameter is a variable representing user's predefinedemotions (e.g., positive, negative, happy, excited, joyful, relieved,peaceful, calm, irritated, angry, sad, fury, bored, or the like), andmay be obtained based on the semantic information or height informationof the user voice. The type of emotion parameter may be preset by theuser or the system.

The emotion parameter may be obtained by inputting voice information toa deep learning neural network model. The processor 130 may obtain theemotion parameter for the user voice by inputting user voice informationto the deep learning neural network model such as deep neural network(DNN), convolutional neural network (CNN), recurrent neural network(RNN), or the like.

Referring to FIG. 12, the emotion parameter for the user voice may bedefined in a two-dimensional space including a first axis 1210corresponding to the meaning of the user voice and a second axis 1220corresponding to the height of the user voice.

The processor 130 may determine the height of the user voice using theuser voice information. To be specific, the processor 130 may determinethe height of the user voice by analyzing a frequency (or the number ofvibrations) of the received user voice information.

The processor 130 may input the user voice information to the deeplearning neural network model and determine whether the meaning of theuser utterance included in the user voice information is positive,neutral, or negative.

The processor 130 may obtain the emotion parameter for the user voiceusing the semantic information of the user voice obtained from the uservoice information and the height information of the user voice.

For example, when the meaning of the user voice is positive and theheight of the user voice is greater than or equal to the firstpredetermined value, the processor 130 may obtain “excited” as the valueof the emotion parameter. If the meaning of the user voice is negativeand the height of the user voice is less than the second predeterminedvalue, the processor 130 may obtain “sad” as the value of the emotionparameter.

The processor 130 may determine the understanding level of the manualbased on the obtained emotion parameter. For example, when the emotionparameter has a predetermined first value (e.g., positive, joyful,excited, or the like), the processor 130 may determine that theunderstanding level is a high level. In addition, when the emotionparameter has a preset second value (e.g., negative, angry, sad, or thelike), the processor 130 may determine that the understanding level is alow level. In addition, the processor 130 may determine that theunderstanding level is a neutral level when the emotion parameter has apredetermined third value or the value is not a predetermined firstvalue and a second value.

Although the emotion parameters are described as being obtained by thesemantic information and the height information of the user voiceinformation in FIG. 12, it is not limited thereto. The emotion parametermay be obtained by one of the semantic information and the heightinformation of the user voice information. For example, by using onlythe semantic information of the user voice information, the processor130 may determine that the understanding level is a high level when thesemantic information is “positive,” understanding level is a neutrallevel when the semantic information is “neutral,” and understandinglevel is a low level when the semantic information is “negative.”

The processor 130 may determine the understanding level for the manualusing the emotion parameter obtained through the user voice information,determine a substitute manual based on the determined understandinglevel, and determine a reproduction starting point of the determinedsubstitute manual.

FIGS. 13 and 14 are views to describe an electronic apparatus fordetermining a reproduction starting point of a substitute manualaccording to various embodiments of the disclosure.

FIG. 13 is a view to describe an electronic apparatus for determining areproduction starting point of a substitute manual using stateinformation of an external device according to an embodiment of thedisclosure.

Referring to FIG. 13, the substitute manual determined by the processor130 may include at least one frame corresponding to the plurality ofsteps. Although FIG. 13 illustrates one frame corresponding to each stepfor convenience, each step may include at least one frame. For example,a manual related to an air conditioner filter cleaning method mayinclude at least one frame 1310 corresponding to a guiding step, atleast one frame 1320 corresponding to a step in which a “filtercleaning” warning is displayed on a display of the air conditioner, atleast one frame 1330 corresponding to the filter separation step, atleast one frame 1340 corresponding to the filter cleaning step, and atleast one frame 1350 corresponding to the filter drying step.

In addition, as described above in FIG. 8, the memory 120 may storeinformation on a plurality of sections corresponding to a plurality ofsteps included in the manual. For example, in the memory 120, withrespect to the manual related to the air conditioner filter cleaningmethod, the information on each section that each section of the manualincludes guiding, displaying the filter cleaning warning, filterseparation, filter cleaning, and filter drying and information on thereproduction point of each section, or the like, may be stored.

The processor 130 may receive the state information of the externaldevice from the external device corresponding to the manual anddetermine the state of the external device.

The processor 130 may determine a section including the state of theexternal device among the plurality of sections (or steps) included inthe substitute manual, based on information on a plurality of sectionsof the substitute manual stored in the memory 120. For example, theprocessor 130 may receive the state information that the filter 1301 ofthe air conditioner 1300 is removed from the air conditioner 1300, andmay determine that the section including the state in which the filterof the air conditioner is removed, from among the plurality of stepsincluded in the substitute manual, is the section including the frame1330, based on the information on the plurality of sections of thesubstitute manual.

The processor 130 may determine the section next to the sectioncorresponding to the determined state of the external device as thereproduction starting point of the substitute manual. That is, theprocessor 130 may determine the section including the frame 1340 whichis the next section of the section including the frame 1330 as thereproduction starting point of the substitute manual.

The processor 130 may transmit information about the reproductionstarting point of the substitute manual and the substitute manual to thedisplay device 200, and the display device 200 may reproduce thesubstitute manual based on the received reproduction starting pointinformation. According to the embodiment of FIG. 13, a display devicemay reproduce a manual from a section including a frame 1340, which is areproduction starting point, while skipping a section including frames1310, 1320, and 1330 among a plurality of sections of the substitutemanual.

FIG. 14 is a view to describe an electronic apparatus for determining areproduction starting point of a manual based on a reproduction historyof a manual according to an embodiment of the disclosure.

Referring to FIG. 14, similar to FIG. 13, the substitute manualdetermined by the processor 130 may include a plurality of steps, andinclude at least one frame corresponding to each of the plurality ofsteps.

The memory 120 may store information on a plurality of sectionscorresponding to a plurality of steps included in the manual. Theinformation on a plurality of sections may include information relatedto the number of reproductions of the plurality of sections.

The steps included in the manual may be the same, even if the manualsare different. For example, “a manual related to a voice recognitionsetting method of an air conditioner” and “a manual related to a methodfor setting a voice recognition of a TV” are different manuals, but bothmanuals may include “a pairing step using a remote controller.” In thiscase, information on the number of reproduction times per section of themanual stored in the memory 120 may include the number of timesreproduced through another manual. The number of reproductions of thesection included in the manual may be calculated independently of thenumber of reproductions of the manual and stored. For example, even ifit is stored that the manual related to the voice recognition settingmethod of the air conditioner has been reproduced once, the sectioncorresponding to the “pairing step using the remote controller” includedin the manual related to the method for setting the voice recognition ofthe air conditioner may be stored as being reproduced for three times.

The processor 130 may determine the substitute manual based on thenumber of times of reproduction for each section of the manual stored inthe memory 120. When the number of times of reproduction of each sectionof the manual is greater than or equal to a preset value, the processor130 may replace a manual for a section of which the number of times ofreproduction is greater than or equal to a preset value as a simplemanual and provide the same to the display device 200.

For example, it may be assumed that the manual related to the voicerecognition setting method for an air conditioner is divided into astart guide section 1410, a section for setting pairing with a remotecontroller 1420, a section for pressing a voice recognition button in aremote controller 1430, and the section 1420 has been reproduced for thepredetermined number of times or more.

In the display device 200, in a state where a section corresponding tothe section 1410 is reproduced and a substitute manual according to anunderstanding level is selected, the processor 130 can determine asection 1420 which has been reproduced for greater than or equal to apredetermined number of times, among the plurality of sections 1410,1420, and 1430 of the manual. In addition, the processor 130 may set thenext section of the determined section as the starting section of thesubstitute manual, and determine the starting point of the set startsection as the reproduction starting point of the substitute manual.

Referring to FIGS. 8 to 14, it has been described that the electronicapparatus 100 provides a manual to the display device 200 based on userresponse information received from the user terminal device 300, but itis not necessarily limited thereto. Specifically, according to anotherembodiment, the electronic apparatus 100 may receive a manual requestcommand from the display device 200 and transmit a manual to the displaydevice 200 in accordance with the received manual request command.

FIGS. 15, 16, and 17 are views to describe an electronic apparatus forreceiving a manual transmission command from the display device 200according to various embodiments of the disclosure. FIG. 15 is asequence view to describe a manual transmission process between theelectronic apparatus 100 and the display device 200 according to anembodiment of the disclosure.

Referring to FIG. 15, the processor 130 may generate graphic userinterface (GUI) information of a manual provided to the display device200 in operation S1510. The processor 130 may generate GUI informationcorresponding to the manual by using information on each section of themanual stored in the memory 120. The GUI information may include imageinformation included in the GUI, text information, location informationon which the GUI is displayed, time information of a manual displayedwith the GUI, manual information corresponding to the GUI, or the like.

The processor 130 may transmit the generated GUI information and contentto the display device 200 in operation S1520, and the display device 200may display the GUI corresponding to the GUI information with thecontent using the received GUI information in operation S1530. Thedisplay device 200 may generate a GUI corresponding to GUI informationusing image information and text information among the received GUIinformation, and may display the GUI together with the manual inconsideration of the information on the location where the GUI isdisplayed and the time information of the manual, or the like.

The display device 200 may receive an input for selecting a GUI from theuser 10 in operation S1540. The user 10 may select the GUI through touchinput to the display, or may select the GUI through a separate inputinterface, such as a button. The user 10 may also select a GUI throughan input interface of a remote controller (not shown) for the displaydevice 200.

The display device 200, when receiving a user input for selecting a GUI,may request a manual corresponding to the GUI selected by the user inputin operation S1550. The display device 200 may transmit information on amanual corresponding to the GUI included in the GUI information to theelectronic apparatus 100.

The processor 130 may search a manual corresponding to the GUI inresponse to a request command received from the display device 200 inoperation S1560, and transmit the manual to the display device 200 inoperation S1570. Based on the manual information corresponding to theGUI received from the display device 200, the processor 130 may search amanual corresponding to the GUI and transmit the searched manual to thedisplay device 200.

The display device 200 may reproduce the received manual in operationS1580.

The GUI may correspond to a variety of manuals. The GUI may correspondto a specific section of a general manual, a detailed manual, and asimple manual, but may correspond to a magnified manual in which aspecific area of the manual is magnified or a reduction manual in whicha specific area of the manual is reduced.

FIGS. 16 and 17 are views to describe an electronic apparatus fortransmitting a manual corresponding to the GUI displayed on the displaydevice according to various embodiments of the disclosure.

Referring to FIG. 16, if it is determined that a manual associated witha speech recognition method of an air conditioner is divided into afirst section including a frame 1610, a second section including a frame1620, and a third section including a frame 1630, and that there is apairing detail image of pairing with respect to the second section, theprocessor 130 may generate GUI information requesting a pairing detailimage related to the second section and transmit the GUI information tothe display device 200.

If the display device 200 receives a user input selecting a GUI 1621displayed on the display device 200 based on the GUI information, theprocessor 130 may receive a manual request command corresponding to theGUI from the display device 200 and provide the pairing detail imageassociated with the second section to the display device 200.

In this case, the display device 200 may reproduce a manual 1600corresponding to the selected GUI instead of the second sectionincluding a frame 1620.

Referring to FIG. 17, the processor 130 may generate GUI informationcorresponding to an enlarged image corresponding to a specific sectionof the manual, such as frame 1710 labeling the buttons and otherfeatures of an air condition, and transmit the GUI information to thedisplay device 200. Referring to FIG. 17, while the GUI corresponding tothe enlarged image is displayed, the GUI is not necessarily limitedthereto, and the GUI may be a GUI corresponding to the reduced imagereproduction, a GUI corresponding to the image reproduction of anotherview, or the like.

When determining that there is a magnified image with respect to a frame1710 included in the manual, the processor 130 may generate GUIinformation for requesting a magnified image of the frame 1710 andtransmit the information to the display device 200.

When the display device 200 receives a user input selecting a GUI 1711corresponding to a request for an enlarged image, the display device 200may transmit an enlarged image request command corresponding to theselected GUI to the electronic apparatus 100, and the processor 130 maytransmit the manual according to the received request command to thedisplay device 200.

This is merely exemplary, and if the display device 200 selects a GUIcorresponding to the enlarged image, the display device 200 may enlargeand display the manual displayed on the display device 200 as shown inframe 1720. The display device 200 may not receive the manualcorresponding to the GUI from the electronic apparatus 100, but maymagnify the provided manual and provide the same.

FIG. 18 is a sequence view including an electronic apparatus accordingto an embodiment of the disclosure. Specifically, FIG. 18 is a sequenceview to describe an electronic apparatus for transmitting a manual forthe display device 200.

Referring to FIG. 18, the user terminal device 300 may receive uservoice information from the user 10 in operation S1810. The user voiceinformation may include user voice information related to an externaldevice 500.

The user terminal device 300 may transmit the received user voiceinformation to the electronic apparatus 100 in operation S1820. In thiscase, the processor 130 may perform speech recognition on the user voiceusing a dialog system pre-stored in the memory 120 in operation 51830.The processor 130 may obtain external device information related to uservoice information as a result of voice recognition on the voice of theuser, and may determine an external device based on information relatedto the obtained external device in operation S1840.

The processor 130 may request use history information of the externaldevice 500 to the determined external device 500 in operation S1850, andreceive the use history information of the external device 500 from theexternal device 500 in operation S1860.

The processor 130 may select one of a plurality of manuals related tothe external device 500 based on the received use history information inoperation S1870. The selected manual may be related to the user voiceinformation.

For example, if the user says “the suction force of the robot cleaner isweak”, the processor 130 may determine that the user utterance isassociated with the robot cleaner and receive the usage historyinformation of the robot cleaner from the robot cleaner. The processor130 may select a manual associated with a filter substitution method ofthe robot cleaner based on the received usage history information of therobot cleaner.

The processor 130 may transmit the selected manual to the display device200 in operation S1880, and the display device 200 may reproduce thereceived manual in operation S1890.

FIG. 19 is a view to describe an electronic apparatus according to anembodiment of the disclosure.

Referring to FIG. 19, the electronic apparatus 100 may directly providea manual to a user. The electronic apparatus 100 may directly receive auser voice that is uttered while the manual is being reproduced.

Upon receiving a user voice, the electronic apparatus 100 maypre-process a user voice and convert the voice to a user voice signal.

The electronic apparatus 100 may include a dialog system 400 to performspeech recognition on a voice signal, convert the user voice to a textusing an automatic speech recognition module included in the dialoguesystem, analyze the meaning of the user voice using a natural languageunderstanding module, and obtain a user's understanding level for themanual provided based on the height information and semantic informationof the user voice.

The electronic apparatus 100 may select a substitute manual inconsideration of the type of manual being reproduced and the userunderstanding level, search the selected substitute manual in the memory120, or request the substitute manual to an external device (not shown)such as a content server.

The electronic apparatus 100 may determine the reproduction point of thesubstitute manual and reproduce the section corresponding to thedetermined reproduction point of the substitute manual, in considerationof the user voice information, reproduction history information of thesubstitute manual, state information of an external device, or the like.

The electronic apparatus 100 may be implemented with various types ofelectronic apparatuses. For example, the electronic apparatus 100 may beimplemented as, for example, smartphones, tablet PCs, mobile phones,video telephones, electronic book readers, desktop PCs, laptop PCs,netbook computers, workstations, servers, a personal digital assistant(PDA), a portable multimedia player (PMP), an MP3 player, a medicaldevice, a camera, or a wearable device.

FIG. 20 is a block diagram illustrating an electronic apparatusaccording to an embodiment of the disclosure.

Referring to FIG. 20, the electronic apparatus 100 may include thecommunication interface 110, the memory 120, the input interface 140,the display 150, a speaker 160, a camera 170, and the processor 140. Thedescriptions of the communication interface 110, the memory 120, and theprocessor 130 shared with FIG. 2 will not be further described.

The input interface 140 may receive user input to control the electronicapparatus 100. The input interface 140 may receive user voiceinformation for the reproduced content, or receive a user input forselecting a GUI displayed in the display. As illustrated in FIG. 20, theinput interface 140 may include a microphone 141 for receiving an inputof a user voice, a touch panel 142 for receiving a user touch using userhand or a stylus pen, and a button 143 for receiving a usermanipulation, or the like. However, an example of the input interface140 as illustrated in FIG. 20 is merely an example, and another inputinterface (e.g., keyboard, mouse, motion inputter, or the like) may beimplemented.

The display 150 is a configuration to display an image and may beimplemented as, for example, a liquid crystal display (LCD), andcathode-ray tube (CRT), plasma display panel (PDP), organic lightemitting diodes (OLED), transparent OLED (TOLED) for some cases.

The display 150 may display various information according to a controlof the processor 130. The display 150 may display one of a plurality ofmanuals related to an external device. In addition, the display 150 maygenerate a GUI related to a manual. The display 150 may be implementedas a touch screen along with the touch panel 142.

The speaker 160 is a configuration to output not only various audio dataprocessed as decoding, amplification, and noise filtering but alsovarious notification sounds or voice message. The speaker 160 may outputa response to the user inquiry as a voice message in a natural languageformat. The configuration to output audio may be implemented as aspeaker, but this is an example and may be implemented as an outputterminal for outputting audio data.

The camera 170 is configured to photograph an image. The camera 170 canbe installed at the top, bottom, or left and right sides of the display150 so that a user viewing the screen displayed on the display 150 canbe photographed. In addition, the camera 170 may be additionallyinstalled on at least one of the front surface, the rear surface, theleft side surface, or the right side surface, so as to photograph auser's attitude, operation, or the like.

The computer instructions for performing the processing operations ofthe electronic apparatus 100 according to the various embodimentsdescribed above may be stored in a non-transitory computer-readablemedium. The computer instructions stored in this non-transitorycomputer-readable medium may cause the above-described specific deviceto perform the processing operations in the electronic apparatus 100according to the above-described various example embodiments whenexecuted by the processor of the device.

The non-transitory computer readable medium refers to a medium thatstores data semi-permanently rather than storing data for a very shorttime, such as a register, a cache, a memory or etc., and is readable byan apparatus. The aforementioned various applications or programs may bestored in the non-transitory computer readable medium, for example, acompact disc (CD), a digital versatile disc (DVD), a hard disc, aBlu-ray disc, a universal serial bus (USB), a memory card, a read onlymemory (ROM), and the like, and may be provided.

While the disclosure has been shown and described with reference tovarious embodiments thereof, it will be understood by those skilled inthe art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the disclosure as definedby the appended claims and their equivalents.

What is claimed is:
 1. A manual providing method of an electronicapparatus, the method comprising: providing a display device with one ofa plurality of manuals for an external device, wherein the plurality ofmanuals incudes a general manual, a detailed manual, and a simplemanual; obtaining a user response comprising user voice information anduser image information for the provided manual, wherein the user voiceinformation includes voice information uttered while listening to orviewing the provided manual; determining an understanding level for theprovided manual based on the obtained user response, wherein theunderstanding level is: a high level based on a voice height of the uservoice information being greater than or equal to a predetermined firstvalue, a low level based on the voice height of the user voiceinformation being less than a predetermined second value, and a neutrallevel based on the voice height of the user voice information beinggreater than or equal to the predetermined second value; substitutingthe provided manual with another manual among the plurality of manualsaccording to the understanding level, wherein the substituting theprovided manual with another manual comprises: substituting the manualwith the detailed manual when the understanding level is the low leveland the manual corresponds to the general manual, and substituting themanual with the general manual or the detailed manual when theunderstanding level is the low level and the manual corresponds to thesimple manual, receiving state information of the external device fromthe external device; and determining a reproduction starting point inthe substituted manual based on a recognition result of the user voiceinformation included in the user response or the state information,wherein the determining the reproduction starting point comprises: inresponding to the understanding level being the low level, setting thereproduction starting point to a section in the substituted manualcorresponding to a last reproduced section of the provided manual, andin responding to the understanding level being the neutral level or thehigh level, setting the reproduction starting point to a section in thesubstituted manual corresponding to the section that follows the mostrecently reproduced section of the provided manual.
 2. The method ofclaim 1, wherein the general manual, the detailed manual, and the simplemanual are divided into a plurality of sub-contents that aresequentially reproduced.
 3. The method of claim 1, wherein thedetermining of the understanding level comprises: determining anunderstanding level for the manual based on the received stateinformation of the external device.
 4. The method of claim 1, whereinthe determining of the understanding level further comprises: obtainingan emotion parameter from voice information included in the userresponse, and determining the understanding level further based on theemotion parameter.
 5. The method of claim 1, wherein the substituting ofthe provided manual further comprises: streaming data from thedetermined reproduction starting point, among an entire reproductionsection of the substituted manual, to the display device.
 6. The methodof claim 1, wherein the substituting of the provided manual comprisessubstituting the provided manual by providing the display device withaddress information corresponding to the substituted manual andinformation on the reproduction starting point.
 7. An electronicapparatus comprising: a communication interface; a memory; and aprocessor configured to: control the communication interface to providea display device with one of a plurality of manuals for an externaldevice, wherein the plurality of manuals includes a general manual, adetailed manual, and a simple manual, obtain a user response comprisinguser voice information and user image information for the providedmanual, wherein the user voice information includes voice informationuttered while listening to or viewing the provided manual, determine anunderstanding level for the provided manual based on the user response,wherein the understanding level is: a high level based on a voice heightof the user voice information being greater than or equal to apredetermined first value, a low level based on the voice height of theuser voice information being less than a predetermined second value, anda neutral level based on the voice height of the user voice informationbeing greater than or equal to the predetermined second value,substitute the manual with another manual among the plurality of manualsaccording to the understanding level, substitute the manual with thedetailed manual when the understanding level is the low level and themanual corresponds to the general manual, substitute the manual with thegeneral manual or the detailed manual when the understanding level isthe low level and the manual corresponds to the simple manual, receivestate information of the external device from the external device,determine a reproduction starting point in the substituted manual basedon a recognition result of the user voice information included in theuser response or the state information, in responding to theunderstanding level being the low level, set the reproduction startingpoint to a section in the substituted manual corresponding to a lastreproduced section of the provided manual, and in responding to theunderstanding level being the neutral level or the high level, set thereproduction starting point to a section in the substituted manualcorresponding to the section that follows the most recently reproducedsection of the provided manual.
 8. The electronic apparatus of claim 7,wherein the general manual, the detailed manual, and the simple manualare divided into a plurality of sub-contents that are sequentiallyreproduced.
 9. The electronic apparatus of claim 7, wherein theprocessor is further configured to: determine an understanding level forthe manual based on the received state information of the externaldevice.
 10. The electronic apparatus of claim 7, wherein the processoris further configured to: obtain an emotion parameter from voiceinformation included in the user response, and determine theunderstanding level based on the emotion parameter.
 11. The electronicapparatus of claim 7, wherein the processor is further configured tostream data from the determined reproduction starting point, among anentire reproduction section of the substituted manual, to the displaydevice.
 12. The electronic apparatus of claim 7, wherein the processoris further configured to substitute the provided manual by providing thedisplay device with address information corresponding to the substitutedmanual and information on the reproduction starting point.